OP008 - Module Project Peer Review Procedures DRAFT 6.24.24
| OP008 | Module Project Peer Review Procedures |
| Review Committee: Research | Effective Date: 2/08/2010 |
| Attachments: Module Project Template for Peer Review | Revised Date: 06/24/2024 |
| Forms: Review Form | Reviewed Date: 06/24/2024 |
Introduction:
Module projects developed during a grant cycle do not receive external peer review, and therefore the proposers do not have the benefit of a critique by an outsider who has no vested interest in the project. This holds true both for modules developed at the very start of a new grant funding cycle and for ideas for research put forward at a later stage. The following peer review procedures adopted by the TBIMS aim to fill that gap by providing a peer review by experts from within the TBIMS, supplemented as needed/as possible by outside experts. The purpose of the review is to improve the quality of module research, and to optimize use of TBIMS resources.
Purpose:
To describe the process for initiating new module research projects and approving them through an internal peer review process.
Scope:
All TBIMS Centers and TBIMS Follow-up Centers, the NDSC, and NIDILRR.
Responsibilities:
The TBIMS, TBIMS Follow-up Centers, and NDSC will abide by this policy and procedure.
Procedural steps:
Please note that this SOP applies only to modules that have not already been peer reviewed as part of the TBIMS grant application process.
The procedure is as follows:
A proposal for a multi-center module project either emerges from collaborators in at least 2 centers, preferably but not necessarily from a Special Interest Group (SIG), or comes from only one center under special circumstances at the beginning of the cycle. Under the latter circumstances, step 2 of this procedure is skipped.
The idea for the project is sent in draft form via the Notification process used for database projects, with the expectation that every effort will be made to incorporate all centers expressing an interest in collaboration. The NIDILRR Project Officer for each of the participating sites must approve the site’s participation in the module.
The proposal is written using the TBI Model Systems Module Proposal Instructions (attached) and submitted to the Chair of the Research Committee (RC).
The Chair of the RC assigns an RC member to organize a Peer Review Panel. The Panel consists of three reviewers with expertise in the proposal’s topic areas and/or research methodology/statistics. These reviewers should not have contributed to the proposal. If insufficient expertise is available within the TBIMS, outside experts may be invited. The assigned RC member will serve as Panel Chair, but need not be one of the reviewers.
Coordinated by the Panel Chair, panel members independently review the proposal and complete the TBI Model Systems Research Committee Peer Reviewer’s Form (attached). As per the Review Form, they rate each section on a 4-point scale, and give the overall proposal a global rating of 1 to 4, weighing the components as they see fit.
Panel Reviews must be returned to the Panel Chair within 15 business days (3 weeks) of receipt. A 30-minute teleconference is pre-scheduled for the Review Panel for the week following this deadline.
The Panel Chair distributes the reviews to all panel members prior to the teleconference. The teleconference is used to (a) resolve any major discrepancies between reviewers, (b) highlight the most important needs for revisions, if any, and (c) assign a final global score representing the consensus of the panel.
The Panel Chair forwards the reviews, with scores and suggested revisions to the RC Chair, the SIG members who collaborated on the proposal, the NIDILRR TBIMS Program Manager and the NIDILRR Project Officers of the proposing Centers.
At the SIG’s/originating group’s request, the Panel Chair meets by telephone with the PI to answer questions on the review, and supply additional detail. Other Review Panel members and SIG/group members may join this discussion.
The entire process from submission of a proposal to the Research Committee Chair to receipt of a written review by proposer(s) will under normal circumstances take no longer than 2 months.
A global score of 3 or 4 indicates that the project may move forward. The proposal, the reviews with any recommendations for change, and the scores will be forwarded to the SIG members who collaborated on the proposal, the NIDILRR TBIMS Project Manager, and the NIDILRR Project Officers of the proposing Centers, with an indication that the project may proceed. A notification of the approved project will also be sent to the PD list server, so that all centers may be informed of the new module projects.
A score of 1 or 2 reflects a judgment by the review panel that the project as proposed should not go forward. The originators of the proposal may revise and resubmit the proposal to the review panel once. If the score does not exceed 2 on the second try, the review process ends for that project. However, an appeal may be made to the Research Committee if the decision is perceived as inaccurate or unfair. The Research Committee will convene via teleconference call as needed and consider appeals on a case-by-case basis.
Once a project has been recommended and officially approved by NIDILRR, the assignment of a Project Officer, if necessary, will follow within 1-2 weeks.
Compliance:
All TBIMS Centers and TBIMS Follow-up Centers are responsible for adhering to this policy and its procedures.
References:
None
History:
| Date | Action |
|---|---|
| 7/1/2005 | Version of procedure used to create this SOP |
| 1/25/2010 | Transferred to SOP template and revised |
| 2/8/2010 | Approved by Project Directors |
| 2/16/2010 | Added Submission Form and Reviewer Form |
| 7/7/2014 | Added start-of-cycle process and otherwise revised |
| 6/12/2015 | Updated NIDRR to NIDILRR |
| 1/27/2019 | Reviewed |
| 6/24/2024 | Removed section pertaining to beginning-of-cycle modules |
Review schedule:
At least every 5 years.
TBI Model Systems Module Proposal Instructions
Please create a document for the proposed module project with each of the categories below. The length of the completed form should be about 8 pages (start-of-cycle proposal) or 3-5 pages (midcycle proposal) (single spaced). It is not necessary to include formatted references. Please note that this is a minimum length. If you have a longer proposal written for another purpose, e.g., IRB submission, information from it may be included without editing.
Title of project:
PI and lead center:
(Anticipated) Collaborators and their centers:
Period during which proposed project will collect data:
Start date:
End date:
Date this form is submitted:
Background and Significance (knowledge gaps the research is intended to fill; innovative aspects of the research plan):
Research Plan (overview of experimental design, independent and dependent variables, specific aims, hypotheses):
Methods (participants: inclusion/ exclusion and anticipated number [power calculations]; procedure and instruments to be used in data collection [attach data collection forms if available]):
Data Analysis Plan (statistical methods to be used):
Anticipated Outcome(s):
Resources expected to be contributed by a collaborating site. (Describe both one-time, repeated and per-subject resources, in terms of required FTE/hours by category of staff, or dollars; differentiate various scenarios based on different numbers of participating sites):
TBI Model System Research Committee Peer Reviewer’s Form
Please create a document that details your critique of the proposal that has been submitted. Then submit your document to the Panel Chair and fellow panel members.
Title of project:
PI to whom feedback will be given:
Panel chair:
Reviewer:
Date of review:
Please rate on a scale from 1(poor) - 4(excellent), and provide comments on each of the following sections. List separately any flaws in the proposal that you consider to be fatal because of issues of cost, ethics, required number of subjects, nature of or quantity of steps in the protocol, or any other aspect of the research that you think cannot ever be fixed in a protocol revision. If there are no problems that doom the project in that way, write “no fatal flaws”. Finally, submit a global score with the same scale with 1-2 indicating that in your opinion the proposal should not move forward, and 3-4 indicating approval of the proposal.
Background and Significance (importance of knowledge gaps the research is intended to fill; innovative aspects of the research plan, etc.):
Research Plan (comments on experimental design, selection of independent and dependent variables, time points/periods for interventions/data collection, specific aims, hypotheses):
Methods (participants: inclusion/ exclusion and anticipated number [power calculations]; procedure and instruments to be used in data collection):
Data Analysis Plan (statistical methods to be used):
Anticipated Outcome(s) (useability/ transferability of anticipated findings to practice/research/other expected applications):
Fatal flaws:
Comments, if any, on proposer’s estimate of resources needed:
| Background & Significance | 1 2 3 4 |
|---|---|
| Research Plan | 1 2 3 4 |
| Methods | 1 2 3 4 |
| Data Analysis Plan | 1 2 3 4 |
| Anticipated Outcomes | 1 2 3 4 |
| Global Rating | 1 2 3 4 |